Vehicular driving assist system

ABSTRACT

A vehicular driving assist system includes a plurality of cameras disposed at a vehicle and an image processor that processes captured image data. With a trailer equipped with at least one trailer camera hitched to the vehicle, a display screen displays video images derived from captured image data. The system determines a reversing path for the vehicle and trailer to follow to assist the driver in backing up the trailer, and generates a graphic overlay at the display screen that indicates the determined reversing path. During the backing up maneuver of the trailer hitched to the vehicle, and responsive to determination of an object rearward of the trailer, the system (i) determines an adjusted reversing path to avoid the object and (ii) adjusts the graphic overlay to indicate the adjusted reversing path of the vehicle and trailer for the driver to follow in backing up the trailer.

CROSS REFERENCE TO RELATED APPLICATIONS

The present application is a continuation of U.S. patent applicationSer. No. 16/147,946, filed Oct. 1, 2018, now U.S. Pat. No. 10,870,449,which is a continuation of U.S. patent application Ser. No. 15/238,005,filed Aug. 16, 2016, now U.S. Pat. No. 10,086,870, which claims thefiling benefits of U.S. provisional application Ser. No. 62/206,443,filed Aug. 18, 2015, which is hereby incorporated herein by reference inits entirety.

FIELD OF THE INVENTION

The present invention relates generally to a vehicle vision system for avehicle and, more particularly, to a vehicle vision system that utilizesone or more cameras at a vehicle to assist a driver of the vehicle inmaneuvering the vehicle with a trailer.

BACKGROUND OF THE INVENTION

Use of imaging sensors in vehicle imaging systems is common and known.Examples of such known systems are described in U.S. Pat. Nos.5,949,331; 5,670,935 and/or 5,550,677, which are hereby incorporatedherein by reference in their entireties.

In some systems, when attaching a trailer to the vehicle, the driver hasto enter its properties to put the trailer driving aid system into aposition to properly calculate the driving aids overlays, when backingup with a trailer attached. Some more advanced systems are able todetect the trailer length and the distance of the trailer's axle to thehatch by watching the trailer's turning behavior when the vehicle andtrailer are in motion using visual data processing such as described inU.S. Publication No. US-2014-0160276, which is hereby incorporatedherein by reference in its entirety.

Wireless camera data transmission is known, especially BLUETOOTH® andWLAN. To attach wireless cameras onto vehicles or trailers is alsoknown. Analog image transmission (such as NTSC) to an aftermarketdisplay device installed in the vehicle cabin is common use. In U.S.Publication No. US-2014-0340510, which is hereby incorporated herein byreference in its entirety, the integration of a trailer attached(possibly wireless) camera to an OEM vehicle (possibly top view) visionsystem is described including the data transmission architecture and thevisual human interface. A towing vehicle based trailer stability controlis well described in U.S. Publication No. US-2014-0340510.

From German DE 102009057996 (Daimler 2009), it is known how to calibratea trailer rear-mounted camera relative to a trailing vehicle by visualmotion flow analysis via difference image data processing, resulting ina corresponding set of translation and rotation parameters.

In WO 2015/001066, a vehicle trailer back up control system with trailercameras is disclosed that is able to determine a target route (pathplan) with route corridor. Also suggested is 3D representation of thescene. It lacks on describing the according vision processing and cameracalibration to do so.

SUMMARY OF THE INVENTION

The present invention provides a driver assistance system or visionsystem or imaging system for a vehicle that utilizes one or more cameras(preferably one or more CMOS cameras) to capture image datarepresentative of images exterior of the vehicle, with the vehicletowing a trailer that includes one or more cameras, and with the systemproviding enhanced maneuvering of the vehicle and trailer, such as in areversing direction for parking a trailer.

The present invention provides a vehicle and trailer destination orparking lot determination system having a path planning algorithm andvehicle-trailer train control within a vehicle environment processed bya scene understanding system using environmental sensors and sensorfusion algorithms. The vehicle-trailer train control system utilizesvisual trailer identifying and angle detection and processingalgorithms. The vehicle-trailer train path planning uses a two stagepath planner having a geometric planning with consecutive optimalcontrol minimization algorithm. The environmental scene understandingsystem uses classifiers for the sensor clustering and input waging witha consecutive classifier based context determination algorithm.

These and other objects, advantages, purposes and features of thepresent invention will become apparent upon review of the followingspecification in conjunction with the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a plan view of a vehicle with a vision system thatincorporates cameras in accordance with the present invention;

FIG. 2 is an image of a display screen displaying images in accordancewith the present invention;

FIG. 3 is a plan view of a vehicle towing a trailer; and

FIG. 4 is a plan view similar to FIG. 3 , showing paths of the vehicleand trailer during a rearward turning maneuver.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

A vehicle vision system and/or driver assist system and/or objectdetection system and/or alert system operates to capture images exteriorof the vehicle and may process the captured image data to display imagesand to detect objects at or near the vehicle and in the predicted pathof the vehicle, such as to assist a driver of the vehicle in maneuveringthe vehicle in a rearward direction. The vision system includes an imageprocessor or image processing system that is operable to receive imagedata from one or more cameras and provide an output to a display devicefor displaying images representative of the captured image data.Optionally, the vision system may provide display, such as a rearviewdisplay or a top down or bird's eye or surround view display or thelike.

Referring now to the drawings and the illustrative embodiments depictedtherein, a vehicle 10 includes an imaging system or vision system 12that includes at least one exterior facing imaging sensor or camera,such as a rearward facing imaging sensor or camera 14 a (and the systemmay optionally include multiple exterior facing imaging sensors orcameras, such as a forwardly facing camera 14 b at the front (or at thewindshield) of the vehicle, and a sidewardly/rearwardly facing camera 14c, 14 d at respective sides of the vehicle), which captures imagesexterior of the vehicle, with the camera having a lens for focusingimages at or onto an imaging array or imaging plane or imager of thecamera (FIG. 1 ). Optionally, a forward viewing camera may be disposedat the windshield of the vehicle and view through the windshield andforward of the vehicle, such as for a machine vision system (such as fortraffic sign recognition, headlamp control, pedestrian detection,collision avoidance, lane marker detection and/or the like). The visionsystem 12 includes a control or electronic control unit (ECU) orprocessor 18 that is operable to process image data captured by thecamera or cameras and may detect objects or the like and/or providedisplayed images at a display device 16 for viewing by the driver of thevehicle (although shown in FIG. 1 as being part of or incorporated in orat an interior rearview mirror assembly 20 of the vehicle, the controland/or the display device may be disposed elsewhere at or in thevehicle). The data transfer or signal communication from the camera tothe ECU may comprise any suitable data or communication link, such as avehicle network bus or the like of the equipped vehicle.

The present invention provides a driver assist system that assist thedriver in backing up a trailer hooked to the rear of the towing vehicle.The system may utilize aspects of the systems described in U.S. Pat. No.9,085,261 and/or U.S. Publication Nos. US-2015-0042808; US-2015-0042807;US-2015-0042806; US-2015-0002670; US-2014-0336876; US-2014-0160291;US-2014-0160276 and/or US-2014-0067206, and/or International PublicationNo. WO 2014/204794, which are hereby incorporated herein by reference intheir entireties.

Under use of a trailer camera calibration algorithm similar to such assuggested in DE 102009057996, optionally utilizing methods such asdescribed in U.S. Publication No. US-2014-0160276, one or multipletrailer mounted cameras may be calibratable to the vehicle sensorprocessing and vision system of the towing (ego-) vehicle's coordinatesystem, by processing the motion flow visual of the camera or sensor inrelation to the motion flow of the vehicle sensor system. The coordinatesystem may be grid based or may be a vector map or a hybrid with gridmaps for the near area and vector based in further distance (such asdescribed in U.S. Publication No. US-2015-0344028, which is herebyincorporated herein by reference in its entirety). As an additionalaspect of the invention, the vehicle towed trailer camera or cameras oroptional other distance sensor or sensors, such as an ultrasound sensoror ultrasound array sensor, RADAR, Time-of-Flight (TOF) or LIDAR or thelike, may be fused to the vehicle environmental map system as additionalsensor input fused by the (Baysien, Adaboost or other type) classifierin the same manner as the comparable other sensor inputs to the‘Environment and Object map’ of the sensor fusion, such as by utilizingaspects of the systems described in U.S. Publication No.US-2015-0344028. Also similar to U.S. Publication No. US-2015-0344028,the sensor(s) may have a n−1 feedback loop piping in one preceding frameor sensor sample of that same sensor and may have a static inputdescribing the specific sensor properties (at the given point of time n)to the classifier.

In accordance with an aspect of the present invention, although areaswhich are (still) not encompassed by any camera as in the above citedU.S. Publication No. US-2014-0340510 may not be showable on a real timetop view generated image, the mapping distance between the trailercamera (or cameras) image(s) and the images captured by the vehiclecameras may be a plausible distance when using a calibrated trailercamera. The mapping may occur in a machine vision system while the humaninterface may be a top scene view with the ego vehicle and the trailerin it (such as shown in FIGS. 3 and 4 ). FIG. 3 shows the detectedenvironmental scene with the vehicle overlay 30 of the vehicle and thetrailer overlay 31 in it. FIG. 4 shows additionally an overlay of anarea inserted to the real time image by historical data (non-real time)content 32, since it is not captured by any sensor or camera. Overlaysof a planned vehicle reversing path 34 and the trailer's reversing path33 are inserted as well. The trailer and the vehicle may be overlays(augmented) having a true scale relative to the augmented vehicle andthe real environmental scene, while the environmental scene may begenerated non augmented from camera data and optionally other vehiclesensor's data (as discussed as optional under use of the ‘Environmentand Object map’ generation using the sensor processing suggested in U.S.Publication No. US-2015-0344028). Optionally, the areas which are notencompassed by any of the four (or more) surround view vision cameras ortrailer camera (or cameras) may be filled by a non-real time image patchout of the earlier detected scene's map system memory for beingpresented to the driver, such as by using aspects of the systemsdescribed in the above cited U.S. Publication No. US-2014-0160276.Optionally, that non-real time scene content (such as the scene areathat is not in sensor range or that has a too low confidence level (suchas, for example, areas in range of RGB cameras which are in darkness or,for example, ultrasound sensors in front of grass in the sensor rangethat is waving in the wind)) may be highlighted in any suitable manner,such as partially darkening, applying a blinking or colored overlay orby having blinking boundaries, since the non-real time space may be goodfor assuming the drivable (static) space but may lead a driver tooverlook dynamic scene components such as a person or vehicle which mayhave stepped or drove into that maneuvering area which may be hazardousfor that person, such as can be understood with reference to FIG. 4 .

In the customer interface described in U.S. Publication No.US-2014-0340510, and such as shown in FIG. 2 (which corresponds to FIG.40 of U.S. Publication No. US-2014-0340510), it desirable to have somearea between the vehicle rear camera and the trailer camera within thetop view image showing voids or black bars in the top view image displaymode (with vehicle plus trailer) between the areas encompassed by thevehicle cameras and the area encompassed by the trailer camera since thetrailer camera was not calibratable with the vehicle cameras.

Optionally, automated trailer parking or ramp docking DAS may berealized by using an optimal control algorithm under use of a costminimization function, optionally having a two stage path planner havinga first planning stage with a geometrical constructed path planning anda consecutive stage running an optimal control minimization algorithm,which is for smoothing the path and reducing set point extreme values,similar to that suggested in U.S. patent application Ser. No.15/206,801, filed Jul. 11, 2016 and published Jan. 19, 2017 as U.S.Publication No. US-2017-0015312, and/or U.S. Publication No.US-2015-0344028, which are hereby incorporated herein by reference intheir entireties. Known trailer assistant systems allow the user tonavigate a trailer (that is hooked on a propelling or driven vehicle)rearwardly in a curvature controlled manner. In general, trailers behaveunstably when being pushed rearwardly. Also, skilled drivers arechallenged when backing up vehicle-trailer-trains. Curvature controllingtrailer assist systems ease the task by controlling the vehicle'ssteering wheels in a manner that the trailer is describing a curve in aradius actually set by the user's input, such as described in U.S.Publication No. US-2012-0271515.

It is known that the trailer's path and/or the vehicle's path accordingto the actual set point is augmented on the vehicle's vision system,such as, for example, via a top view vision system. Since naturally therear vehicle camera is mostly blocked by the trailer which is beingtowed by the vehicle, the path to the area the trailer is moving towardsis not visible by the vehicle rear camera. To overcome this matter, someOEMs offer the solution that the vehicle vision is transferred to asmart phone and also the vehicle's curvature and longitudinal control isimplemented on the smart phone. The user sets the curvature setpoint ofthe trailer and steers the speed of the vehicle-trailer train. Since theuser can exit the vehicle and watch the scene from aside, possiblyinvisible to any towing vehicle camera, he or she can avoid backing thetrailer into a collision or harming someone.

DE 000010128792 discloses a collision avoidance and warning system ofdrive trains with trailers for maneuvering forward or backwardly. Inthere it is disclosed that required driving space for the towing vehicleand the trailer all together gets predicted. That system's predictiondoes not include trajectory planning, the prediction is limited to onecircle segment and is based on holding the actual trailer steeringangle. The reality is different when maneuvering a trailer forwardly orrearwardly in curves for reaching a specific spot or for avoidingobstacles.

For easier prediction of the last trailer's path of multi-link trains,the curves can be planned in circle segments optionally. In DE102008043675, a control for controlling the rearward traveling path ofmultiple links attached to a leading towing vehicle with just the towingvehicle's front wheels controllable was suggested. The controlmathematics were more or less poor or not executable in practice since acouple of correction constants were to be determined. Nonholonomicmotion path planning of n-trailers by using the Gourset normal form wasdiscussed in IEEE Transactions on Automatik Control, Vol. 40, No. 5, May1995. Another approach capable to avoid dynamic obstacles was shown byLamiraux in Springer Path Optimization for Nonholonomic Systems:Application to Reactive Obstacle Avoidance and Path Planning, 2003.

The vehicle-trailer train path planner of the present invention mayutilize one of the path planning strategies as described in the abovecited references or one similar to the planner and control recentlypublished in International Publication No. WO 2015/001066. InternationalPublication No. WO 2015/001066 though may lack executability by a personskilled in the subject since it does not include any disclosure on howto calibrate the trailer cameras or sensors to the vehicle system sothat the control can steer the trailer along a designated pathautomatically. Without calibration, the scene detection may fail, thepath planning, which is depending on a correct scene detection, may failand the maneuver execution may fail because the path will not match tothe real scene, although the path planner itself may work correctly (butusing wrong input data).

Departing from the prior art, including International Publication No. WO2015/001066, the system of the present invention uses the trailer cameranot just to determine that the trailer is approaching possibly hazardousobjects but to assimilate the visual scene information to the vehicle'sscene sensor processing (master-) system. The trailer cameras arecalibrated essentially by motion flow (such as to correct for improperinstallation, such as being tilted, rolled or yawed or being offcenter), such as similar to the calibration of the surround visionsystem's fish eye cameras (14 a, 14 b, 14 c and 14 d) attached to the(towing) vehicle such as shown in FIG. 1 . The calibration may requirethe vehicle-trailer train to be in motion, for having motion flow. Thescenery may require some texture or pattern so that the system has somefix points to track and motion assess. The texture or pattern should benaturally given when the train is driven in natural environment (such asgiven by trees, buildings, vehicles, and/or the like in the environmentsurrounding the vehicle and trailer). The initial test drive forinitializing the trailer camera in coordinates to the tow vehicle borncoordinate (map-) system may be comparably short, such as about 10 m orthereabouts. The additionally required trailer angle detection systemmay be (towing-) vehicle rear camera based or vehicle rear endultrasound sensor (possibly ultrasound sensor array) based or may beanother direct mounted angle sensor based or contactless sensor basedsystem capable of detecting the trailer angle against the towingvehicle, optionally having a dedicated target (sticker) on the trailerto enable the trailer angle detection system to function, or optionallytargetless trailer detection may be incorporated in the automatedtrailer path control system of the present invention.

Optionally, the trailer path control system of the present invention mayadditionally be capable of avoiding hazards with dynamic objects such aspedestrians walking through the pursued trailer destination or parkingspot or the path to it. The system's algorithm may have a threshold timewhich alters the classification of a moving object to a static objectwhen it comes to a stop over a certain time (such as, for example, whenanother vehicle within the environmental scene [map] stops). While thesystem may just wait until a moving object, such as, for example, apedestrian or the like, has left the trailer train destination path, thesystem may optionally redo the path planning in case the static map haschanged, such as when another vehicle has occupied (and stopped) a partof the free space that was part of the destination path originally.

The system may utilize aspects of the trailering or trailer angledetection systems described in U.S. Pat. Nos. 9,086,261 and/or6,690,268, and/or U.S. Publication Nos. US-2015-0217693;US-2014-0160276; US-2014-0085472 and/or US-2015-0002670, and/or U.S.provisional applications, Ser. No. 62/303,546, filed Mar. 4, 2016,and/or Ser. No. 62/290,167, filed Feb. 2, 2016, which are herebyincorporated herein by reference in their entireties.

The camera or sensor may comprise any suitable camera or sensor.Optionally, the camera may comprise a “smart camera” that includes theimaging sensor array and associated circuitry and image processingcircuitry and electrical connectors and the like as part of a cameramodule, such as by utilizing aspects of the vision systems described inInternational Publication Nos. WO 2013/081984 and/or WO 2013/081985,which are hereby incorporated herein by reference in their entireties.

The system includes an image processor operable to process image datacaptured by the camera or cameras, such as for detecting objects orother vehicles or pedestrians or the like in the field of view of one ormore of the cameras. For example, the image processor may comprise animage processing chip selected from the EYEQ family of image processingchips available from Mobileye Vision Technologies Ltd. of Jerusalem,Israel, and may include object detection software (such as the typesdescribed in U.S. Pat. Nos. 7,855,755; 7,720,580 and/or 7,038,577, whichare hereby incorporated herein by reference in their entireties), andmay analyze image data to detect vehicles and/or other objects.Responsive to such image processing, and when an object or other vehicleis detected, the system may generate an alert to the driver of thevehicle and/or may generate an overlay at the displayed image tohighlight or enhance display of the detected object or vehicle, in orderto enhance the driver's awareness of the detected object or vehicle orhazardous condition during a driving maneuver of the equipped vehicle.

The vehicle may include any type of sensor or sensors, such as imagingsensors or radar sensors or lidar sensors or ladar sensors or ultrasonicsensors or the like. The imaging sensor or camera may capture image datafor image processing and may comprise any suitable camera or sensingdevice, such as, for example, a two dimensional array of a plurality ofphotosensor elements arranged in at least 640 columns and 480 rows (atleast a 640×480 imaging array, such as a megapixel imaging array or thelike), with a respective lens focusing images onto respective portionsof the array. The photosensor array may comprise a plurality ofphotosensor elements arranged in a photosensor array having rows andcolumns. Preferably, the imaging array has at least 300,000 photosensorelements or pixels, more preferably at least 500,000 photosensorelements or pixels and more preferably at least 1 million photosensorelements or pixels. The imaging array may capture color image data, suchas via spectral filtering at the array, such as via an RGB (red, greenand blue) filter or via a red/red complement filter or such as via anRCC (red, clear, clear) filter or the like. The logic and controlcircuit of the imaging sensor may function in any known manner, and theimage processing and algorithmic processing may comprise any suitablemeans for processing the images and/or image data.

For example, the vision system and/or processing and/or camera and/orcircuitry may utilize aspects described in U.S. Pat. Nos. 9,233,641;9,146,898; 9,174,574; 9,090,234; 9,077,098; 8,818,042; 8,886,401;9,077,962; 9,068,390; 9,140,789; 9,092,986; 9,205,776; 8,917,169;8,694,224; 7,005,974; 5,760,962; 5,877,897; 5,796,094; 5,949,331;6,222,447; 6,302,545; 6,396,397; 6,498,620; 6,523,964; 6,611,202;6,201,642; 6,690,268; 6,717,610; 6,757,109; 6,802,617; 6,806,452;6,822,563; 6,891,563; 6,946,978; 7,859,565; 5,550,677; 5,670,935;6,636,258; 7,145,519; 7,161,616; 7,230,640; 7,248,283; 7,295,229;7,301,466; 7,592,928; 7,881,496; 7,720,580; 7,038,577; 6,882,287;5,929,786 and/or 5,786,772, and/or U.S. Publication Nos. 2014/0340510;2014/0313339; 2014/0347486; 2014/0320658; 2014/0336876; 2014/0307095;2014/0327774; 2014/0327772; 2014/0320636; 2014/0293057; 2014/0309884;2014/0226012; 2014/0293042; 2014/0218535; 2014/0218535; 2014/0247354;2014/0247355; 2014/0247352; 2014/0232869; 2014/0211009; 2014/0160276;2014/0168437; 2014/0168415; 2014/0160291; 2014/0152825; 2014/0139676;2014/0138140; 2014/0104426; 2014/0098229; 2014/0085472; 2014/0067206;2014/0049646; 2014/0052340; 2014/0025240; 2014/0028852; 2014/005907;2013/0314503; 2013/0298866; 2013/0222593; 2013/0300869; 2013/0278769;2013/0258077; 2013/0258077; 2013/0242099; 2013/0215271; 2013/0141578and/or 2013/0002873, which are all hereby incorporated herein byreference in their entireties. The system may communicate with othercommunication systems via any suitable means, such as by utilizingaspects of the systems described in International Publication Nos. WO2010/144900; WO 2013/043661 and/or WO 2013/081985, and/or U.S. Pat. No.9,126,525, which are hereby incorporated herein by reference in theirentireties.

The imaging device and control and image processor and any associatedillumination source, if applicable, may comprise any suitablecomponents, and may utilize aspects of the cameras (such as variousimaging sensors or imaging array sensors or cameras or the like, such asa CMOS imaging array sensor, a CCD sensor or other sensors or the like)and vision systems described in U.S. Pat. Nos. 5,760,962; 5,715,093;6,922,292; 6,757,109; 6,717,610; 6,590,719; 6,201,642; 5,796,094;6,559,435; 6,831,261; 6,822,563; 6,946,978; 7,720,580; 8,542,451;7,965,336; 7,480,149; 5,550,677; 5,877,897; 6,498,620; 5,670,935;5,796,094; 6,396,397; 6,806,452; 6,690,268; 7,005,974; 7,937,667;7,123,168; 7,004,606; 6,946,978; 7,038,577; 6,353,392; 6,320,176;6,313,454 and/or 6,824,281, and/or International Publication Nos. WO2009/036176; WO 2009/046268; WO 2010/099416; WO 2011/028686 and/or WO2013/016409, and/or U.S. Pat. Publication Nos. US 2010-0020170 and/orUS-2009-0244361, which are all hereby incorporated herein by referencein their entireties.

Optionally, the vision system may include a display for displayingimages captured by one or more of the imaging sensors for viewing by thedriver of the vehicle while the driver is normally operating thevehicle. Optionally, for example, the vision system may include a videodisplay device disposed at or in the interior rearview mirror assemblyof the vehicle, such as by utilizing aspects of the video mirror displaysystems described in U.S. Pat. No. 6,690,268 and/or U.S. Publication No.US-2012-0162427, which are hereby incorporated herein by reference intheir entireties. The video mirror display may comprise any suitabledevices and systems and optionally may utilize aspects of the compassdisplay systems described in U.S. Pat. Nos. 7,370,983; 7,329,013;7,308,341; 7,289,037; 7,249,860; 7,004,593; 4,546,551; 5,699,044;4,953,305; 5,576,687; 5,632,092; 5,677,851; 5,708,410; 5,737,226;5,802,727; 5,878,370; 6,087,953; 6,173,508; 6,222,460; 6,513,252 and/or6,642,851, and/or European patent application, published Oct. 11, 2000under Publication No. EP 0 1043566, and/or U.S. Publication No.US-2006-0061008, which are all hereby incorporated herein by referencein their entireties. Optionally, the video mirror display screen ordevice may be operable to display images captured by a rearward viewingcamera of the vehicle during a reversing maneuver of the vehicle (suchas responsive to the vehicle gear actuator being placed in a reversegear position or the like) to assist the driver in backing up thevehicle, and optionally may be operable to display the compass headingor directional heading character or icon when the vehicle is notundertaking a reversing maneuver, such as when the vehicle is beingdriven in a forward direction along a road (such as by utilizing aspectsof the display system described in International Publication No. WO2012/051500, which is hereby incorporated herein by reference in itsentirety).

Optionally, the vision system (utilizing the forward facing camera and arearward facing camera and other cameras disposed at the vehicle withexterior fields of view) may be part of or may provide a display of atop-down view or birds-eye view system of the vehicle or a surround viewat the vehicle, such as by utilizing aspects of the vision systemsdescribed in International Publication Nos. WO 2010/099416; WO2011/028686; WO 2012/075250; WO 2013/019795; WO 2012/075250; WO2012/145822; WO 2013/086249 and/or WO 2013/109869, and/or U.S.Publication No. US-2012-0162427, which are hereby incorporated herein byreference in their entireties.

Changes and modifications in the specifically described embodiments canbe carried out without departing from the principles of the invention,which is intended to be limited only by the scope of the appendedclaims, as interpreted according to the principles of patent lawincluding the doctrine of equivalents.

The invention claimed is:
 1. A vehicular driving assist system, saidvehicular driving assist system comprising: a plurality of vehiclecameras disposed at a vehicle and having respective fields of viewexterior of the vehicle, wherein said plurality of vehicle camerasincludes a rear backup camera disposed at a rear portion of the vehicleand viewing at least rearward of the vehicle; at least one trailercamera disposed at a trailer hitched to the vehicle and having arespective field of view exterior of the trailer; an image processoroperable to process image data captured by said plurality of vehiclecameras and by said at least one trailer camera; wherein, responsive toprocessing of image data captured by said plurality of vehicle camerasand by said at least one trailer camera during a backing up maneuver ofthe trailer hitched to the vehicle, said vehicular driving assist systemdetermines objects rearward of the trailer that are present in the fieldof view of said at least one trailer camera; a display screen disposedin the vehicle for viewing by a driver of the vehicle, wherein saiddisplay screen displays video images derived from image data captured byat least one camera selected from the group consisting of (i) saidplurality of vehicle cameras and (ii) said at least one trailer camera;wherein said vehicular driving assist system determines a reversing pathfor the vehicle and trailer to follow to assist the driver in backing upthe trailer; wherein said vehicular driving assist system generates agraphic overlay at said display screen overlaying the displayed videoimages, the graphic overlay indicating the determined reversing path forthe driver to follow in backing up the trailer; wherein, during thebacking up maneuver of the trailer hitched to the vehicle along thedetermined reversing path, said vehicular driving assist systemdetermines an object rearward of the trailer; and wherein, during thebacking up maneuver of the trailer hitched to the vehicle along thedetermined reversing path, and responsive to determination of the objectrearward of the trailer via processing of image data captured by atleast said at least one trailer camera, said vehicular driving assistsystem (i) determines an adjusted reversing path to avoid the object and(ii) adjusts the graphic overlay to indicate the adjusted reversing pathof the vehicle and the trailer for the driver to follow in backing upthe trailer.
 2. The vehicular driving assist system of claim 1, whereinsaid vehicular driving assist system controls a steering system of thevehicle to steer the vehicle to back up the trailer along the determinedreversing path.
 3. The vehicular driving assist system of claim 1,wherein, responsive to processing of image data captured by saidplurality of vehicle cameras and by said at least one trailer camerawhile the vehicle is backing up the trailer, said vehicular drivingassist system initializes said at least one trailer camera incoordinates that correspond to coordinates of said plurality of vehiclecameras.
 4. The vehicular driving assist system of claim 1, wherein saidvehicular driving assist system calibrates said at least one trailercamera responsive at least in part to a determination of a trailer angleof a longitudinal axis of the trailer relative to a longitudinal axis ofthe vehicle.
 5. The vehicular driving assist system of claim 4, whereinsaid vehicular driving assist system determines the trailer angleresponsive at least in part to image processing of image data capturedby at least said rear backup camera of said plurality of vehiclecameras.
 6. The vehicular driving assist system of claim 4, wherein saidvehicular driving assist system determines the trailer angle responsiveat least in part to image processing of image data captured by at leastsaid rear backup camera of said plurality of vehicle cameras and imagedata captured by said at least one trailer camera.
 7. The vehiculardriving assist system of claim 1, wherein the at least one trailercamera at least comprises a rearward viewing trailer camera viewing atleast rearward of the trailer.
 8. The vehicular driving assist system ofclaim 1, wherein, responsive to the determination of the object rearwardof the trailer, said vehicular driving assist system generates an alertto alert the driver of the vehicle.
 9. The vehicular driving assistsystem of claim 1, wherein, responsive to determination of the objectrearward of the trailer, said vehicular driving assist system controls abraking system of the vehicle.
 10. The vehicular driving assist systemof claim 1, wherein, responsive to determination of the object rearwardof the trailer, said vehicular driving assist system controls a steeringsystem of the vehicle.
 11. The vehicular driving assist system of claim1, wherein said vehicular driving assist system utilizes a traileridentifying and angle detection algorithm.
 12. The vehicular drivingassist system of claim 1, wherein said vehicular driving assist systemdetermines the reversing path using a two stage path planner.
 13. Thevehicular driving assist system of claim 12, wherein the two stage pathplanner comprises a geometric planning with consecutive optimal controlminimization algorithm.
 14. A vehicular driving assist system, saidvehicular driving assist system comprising: a plurality of vehiclecameras disposed at a vehicle and having respective fields of viewexterior of the vehicle, wherein said plurality of vehicle camerasincludes a rear backup camera disposed at a rear portion of the vehicleand viewing at least rearward of the vehicle; a rearward viewing trailercamera disposed at a rear portion of a trailer hitched to the vehicle,said rearward viewing trailer camera viewing at least rearward of thetrailer; an image processor operable to process image data captured bysaid plurality of vehicle cameras and by said rearward viewing trailercamera; wherein, responsive to processing of image data captured by saidplurality of vehicle cameras and by said rearward viewing trailer cameraduring a backing up maneuver of the trailer hitched to the vehicle, saidvehicular driving assist system determines objects rearward of thetrailer that are present in the field of view of said rearward viewingtrailer camera; a display screen disposed in the vehicle for viewing bya driver of the vehicle, wherein said display screen displays videoimages derived from image data captured by at least one camera selectedfrom the group consisting of (i) said plurality of vehicle cameras and(ii) said rearward viewing trailer camera; wherein said vehiculardriving assist system determines a reversing path for the vehicle andtrailer to follow to assist the driver in backing up the trailer;wherein said vehicular driving assist system generates a graphic overlayat said display screen overlaying the displayed video images, thegraphic overlay indicating the determined reversing path; wherein saidvehicular driving assist system controls a steering system of thevehicle to steer the vehicle to back up the trailer along the determinedreversing path; wherein, during the backing up maneuver of the trailerhitched to the vehicle along the determined reversing path, saidvehicular driving assist system determines an object rearward of thetrailer; and wherein, during the backing up maneuver of the trailerhitched to the vehicle along the determined reversing path, andresponsive to determination of the object rearward of the trailer viaprocessing of image data captured by at least said rearward viewingtrailer camera, said vehicular driving assist system (i) determines anadjusted reversing path to avoid the object, (ii) adjusts the graphicoverlay to indicate the adjusted reversing path of the vehicle and thetrailer and (iii) controls the steering system of the vehicle to steerthe vehicle to back up the trailer along the adjusted reversing path.15. The vehicular driving assist system of claim 14, wherein, responsiveto processing of image data captured by said plurality of vehiclecameras and by said rearward viewing trailer camera while the vehicle isbacking up the trailer, said vehicular driving assist system initializessaid rearward viewing trailer camera in coordinates that correspond tocoordinates of said plurality of vehicle cameras.
 16. The vehiculardriving assist system of claim 14, wherein, responsive to thedetermination of the object rearward of the trailer, said vehiculardriving assist system generates an alert to alert the driver of thevehicle.
 17. The vehicular driving assist system of claim 14, wherein,responsive to determination of the object rearward of the trailer, saidvehicular driving assist system controls a braking system of thevehicle.
 18. A vehicular driving assist system, said vehicular drivingassist system comprising: a plurality of vehicle cameras disposed at avehicle and having respective fields of view exterior of the vehicle,wherein said plurality of vehicle cameras includes a rear backup cameradisposed at a rear portion of the vehicle and viewing at least rearwardof the vehicle; a rearward viewing trailer camera disposed at a rearportion of a trailer hitched to the vehicle, said rearward viewingtrailer camera viewing at least rearward of the trailer; an imageprocessor operable to process image data captured by said plurality ofvehicle cameras and by said rearward viewing trailer camera; wherein,responsive to processing of image data captured by said plurality ofvehicle cameras and by said rearward viewing trailer camera during abacking up maneuver of the trailer hitched to the vehicle, saidvehicular driving assist system determines objects rearward of thetrailer that are present in the field of view of said rearward viewingtrailer camera; wherein said vehicular driving assist system calibratessaid rearward viewing trailer camera responsive at least in part to adetermination of a trailer angle of a longitudinal axis of the trailerrelative to a longitudinal axis of the vehicle; a display screendisposed in the vehicle for viewing by a driver of the vehicle, whereinsaid display screen displays video images derived from image datacaptured by at least one camera selected from the group consisting of(i) said plurality of vehicle cameras and (ii) said rearward viewingtrailer camera; wherein said vehicular driving assist system determinesa reversing path for the vehicle and the trailer to follow to assist thedriver in backing up the trailer; wherein said vehicular driving assistsystem generates a graphic overlay at said display screen overlaying thedisplayed video images, the graphic overlay indicating the determinedreversing path for the driver to follow in backing up the trailer;wherein, during the backing up maneuver of the trailer hitched to thevehicle along the determined reversing path, said vehicular drivingassist system determines an object rearward of the trailer; and wherein,during the backing up maneuver of the trailer hitched to the vehiclealong the determined reversing path, and responsive to determination ofthe object rearward of the trailer via processing of image data capturedby at least said rearward viewing trailer camera, said vehicular drivingassist system (i) determines an adjusted reversing path to avoid theobject and (ii) adjusts the graphic overlay to indicate the adjustedreversing path for the driver to follow in backing up the trailer. 19.The vehicular driving assist system of claim 18, wherein said vehiculardriving assist system controls a steering system of the vehicle to steerthe vehicle to back up the trailer along the determined reversing path.20. The vehicular driving assist system of claim 18, wherein saidvehicular driving assist system determines the trailer angle responsiveat least in part to image processing of image data captured by at leastsaid rear backup camera of said plurality of vehicle cameras.
 21. Thevehicular driving assist system of claim 18, wherein said vehiculardriving assist system determines the trailer angle responsive at leastin part to image processing of image data captured by at least said rearbackup camera of said plurality of vehicle cameras and image datacaptured by said rearward viewing trailer camera.
 22. The vehiculardriving assist system of claim 18, wherein, responsive to thedetermination of the object rearward of the trailer, said vehiculardriving assist system generates an alert to alert the driver of thevehicle.
 23. The vehicular driving assist system of claim 18, wherein,responsive to determination of the object rearward of the trailer, saidvehicular driving assist system controls a braking system of thevehicle.